Validity and reliability of a repeated multi-changes of direction agility test in senior soccer players.

Background: This study aimed to evaluate the validity, reliability, and sensitivity of repeated multi-changes of direction agility test (rMCOD) compared to a soccer-specific field test of repeated sprint ability (S-RSA) and repeated sprint ability test (RSA). Methods: Thirty-five healthy male soccer players (age: 18.4 ± 1.3 years) from Tunisan national soccer league (elite and sub-elite) took part in this study. They performed the tests in a randomized order over five sessions interspaced by at least 72 h. The construct, predictive and discriminant validity, relative and absolute reliability, and sensitivity of the tests were analyzed. The total and best time of the test (the sum for all trials and the trial with the lowest duration on nine, six, and seven attempts for rMCOD, RSA, and S-RSA, respectively), fatigue index, rating of perceived exertion (RPE), and lactate concentration were recorded. Results: rMCOD correlated with both, S-RSA and RSA in total time (r = 0.85 and r = 0.52, respectively) and fatigue index (r = 0.74 and r = 0.83, respectively). Receiver operator characteristics were not able to discriminate between group levels (elite and sub-elite). When comparing training levels, only the fatigue index in S-RSA showed a difference between groups. Fatigue index, total time, and the best time in rMCOD showed excellent reliability, as well as the minimal change detectable (MCD = 0.89, MCD = 0.63, and MCD = 0.11, respectively) was higher than the standard error of the mean (SEM = 0.32, SEM = 0.23, and 0.04, respectively). Conclusion: rMCOD showed large to very large predictive validity compared with the S-RSA and RSA, being a reliable test for the following parameters: the best time and total time to perform the test. Nevertheless, this study design cannot ensure whether or not this test is able to detect real changes in performance in response to training since it did not include a training intervention; besides, rMCOD could not distinguish between elite and sub-elite players, which is a limitation.

Assessing the predictive validity of pre-admission criteria on dental students' academic performance: a cross-sectional study.

BACKGROUND: This study examines the multifaceted factors influencing academic performance among dental students in Saudi Arabia, exploring the complex relationships between demographic, educational, and psychological variables and students' Grade Point Averages (GPAs) to enhance global dental education admission strategies. METHODS: Data on demographics, academic performance indicators (including GPA, high school grades, test scores), accommodation type, parental education, suspension history, and English proficiency were collected through an English-language electronic questionnaire from 900 dental students across multiple institutions in Riyadh. The study employed Pearson's correlation analysis to examine relationships between GPA and various academic and demographic factors. A detailed regression analysis, using a stepwise selection based on Akaike's information criteria (AIC), identified significant GPA predictors and evaluated the average marginal effects of pre-admission variables on GPA, alongside rigorous diagnostic checks to validate the model's robustness. RESULTS: The study revealed a complex interplay of factors impacting GPA among dental students. High school grades, achievement, and aptitude test scores showed moderate positive correlations with GPA, while negative correlations were observed with age and number of times suspended. Regression analysis highlighted age, marital status, high school grades, and test scores as key predictors, with complex interaction effects demonstrating the layered influences of these factors. For instance, the negative impact of age on GPA was moderated by high school grade and aptitude test score. The analysis also highlighted the significant marginal effects of various pre-admission variables on GPA, such as the negative average impact of age and the positive impact of high school grades. CONCLUSIONS: This study provides valuable insights into the multifaceted determinants of academic success in dental education. Our findings underscore the significant roles of high school grades, achievement, and aptitude test scores, as well as the nuanced influence of age and marital status on GPA. These results advocate for a holistic approach in evaluating candidates for dental programs and have broader implications for global dental education, emphasizing the need for comprehensive admission strategies.

Using natural language processing to increase prediction and reduce subgroup differences in personnel selection decisions.

The purpose of this research is to demonstrate how using natural language processing (NLP) on narrative application data can improve prediction and reduce racial subgroup differences in scores used for selection decisions compared to mental ability test scores and numeric application data. We posit there is uncaptured and job-related constructs that can be gleaned from applicant text data using NLP. We test our hypotheses in an operational context across four samples (total N = 1,828) to predict selection into Officer Training School in the U.S. Air Force. Boards of three senior officers make selection decisions using a highly structured rating process based on mental ability tests, numeric application information (e.g., number of past jobs, college grades), and narrative application information (e.g., past job duties, achievements, interests, statements of objectives). Results showed that NLP scores of the narrative application generally (a) predict Board scores when combined with test scores and numeric application information at a level of correlation equivalent to the correlation between human raters (.60), (b) add incremental prediction of Board scores beyond mental ability tests and numeric application information, and (c) reduce subgroup differences between racial minorities and nonracial minorities in Board scores compared to mental ability tests and numeric application information. Moreover, NLP scores predict (a) job (training) performance, (b) job (training) performance beyond mental ability tests and numeric application information, and (c) even job (training) performance beyond Board scores. Scoring of narrative application data using NLP shows promise in addressing the validity-adverse impact dilemma in selection. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Investigating correlations between the PAT and three hand-skill exercises to performance in preclinical laboratory courses.

OBJECTIVES: Perceptual ability test (PAT) is a valid determinant for spatial perceptions. However, proven psychomotor skill tests for dentistry do not currently exist. This study aimed to determine whether performance on the PAT, Operation game, mirror tracing, and wax carving exercises correlate to preclinical laboratory performance in Dental Anatomy and Restorative Dentistry courses. METHODS: First-year dental students (n = 96) participated in the study. Final laboratory grades were obtained from course directors of preclinical Restorative Dentistry and Dental Anatomy courses. Admissions committee provided participants' PAT results. Participants completed a wax carving exercise by carving a cube and a semicircle into a wax block using the wax subtraction method. Two calibrated faculty graded the carvings as Ideal (5), Satisfactory (3), or Unsatisfactory (1). Time to complete the Operation game and number of infractions were recorded. Participants traced the six-pointed star pattern on the Auto Scoring Mirror Tracer instrument in both clockwise and counterclockwise directions. Completion times and number of instances outside the pattern were recorded. Spearman Rank Correlations were used to determine associations at 0.05 confidence interval. RESULTS: Mean PAT score was 21.7; mean times to complete the Operation game and mirror tracing exercise were 4:20 and 1:30, respectively. Mean score on the wax carving exercise was 3.19. Correlations between the independent and dependent variables were minimal to weak. The wax carving exercise was the most reliable predictor of performance. CONCLUSIONS: When PAT scores are striated into low (less than 20), middle (21-22), and high (23-30), it was possible to predict performance in both preclinical laboratory courses.

Do students' personality traits change during medical training? A longitudinal cohort study.

Many medical schools incorporate assessments of personal characteristics, including personality traits, in their selection process. However, little is known about whether changes in personality traits during medical training affect the predictive validity of personality assessments. The present study addressed this issue by examining the stability of personality traits and their predictive validity over a 6-year medical training course. Participants were two cohorts of Swiss medical students (N = 272, 72% of students admitted to Year 2) from whom we collected demographic data, Swiss medical studies aptitude test (EMS) scores, Big Five personality traits scores measured at three times and scores on the multiple-choice and objective structured clinical examination parts of the final medical examination. Our findings indicated that personality traits had medium-to-high rank-order stability (r > .60 over 3 years and r > .50 over 6 years). Mean-level changes were moderate for agreeableness (d = + 0.72) and small for neuroticism and conscientiousness (d = -0.29, d = -0.25, respectively). Individual reliable change indices ranged from 4.5% for openness to 23.8% for neuroticism. The predictive validity was similar to that of the first three years of follow-up. To the best of our knowledge, this is the first study to investigate changes in personality across undergraduate curriculum. Medical students' personality traits were mostly stable across medical school and retain their predictive validity. Consequently, this study supports the use of tools measuring constructs underlying personality traits in selection. In addition, this study confirms that examination formats could favor students with certain personality traits.

Challenging conclusions about predictive bias against Hispanic test takers in personnel selection.

Berry et al. (2020) noted that predictive bias is a function of three factors: subgroup mean difference on the predictor (dx), subgroup mean difference on the criterion (dy), and test validity (rxy). They used meta-analytic estimates of each of these three to examine predictive bias against Hispanic test takers when cognitive tests are used in personnel selection. They found that tests underpredict Hispanic job performance by an average of .21 SDs, which would call into question the fairness of cognitive test use in personnel selection. We located 119 studies in which all three parameters-dy, dx, and rxy-could be obtained, thus holding sample, setting, and operationalization constant in estimating the three parameters within each study. This produced a substantially different conclusion: We find that tests overpredict Hispanic performance by .04-.20 SDs, depending on assumptions made about artifact corrections. Factors contributing to differences between the two studies include differences in range restriction corrections, sample incomparability, and Berry et al.'s use of rxy estimated from the total sample rather than within the majority subgroup. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Evidências de validade do teste informatizado para avaliação das funções executivas / Evidence of validity of the computerized test for evaluating executive functions / Evidencia de validez de la prueba computarizada para evaluación de funciones ejecutivas

Funções executivas (FE) referem-se ao conjunto de habilidades que, de forma integrada, coordenam o comportamento e a cognição. Assim, o comprometimento no desenvolvimento das FE está ligado a vários desfechos negativos ao longo da vida. Portanto, a avaliação dessas habilidades na infância é essencial para identificar e prevenir prejuízos na vida adulta. Este estudo teve como objetivo investigar evidências de validade do Teste Informatizado para Avaliação das Funções Executivas (Tafe) pelo critério de idade e pelo padrão de correlação entre medidas do TAFE e outras medidas de FE. Para isso, foi utilizada uma amostra de 51 crianças, entre 4 e 10 anos de idade, matriculadas em uma escola privada na cidade de Goiânia (GO), da pré-escola ao 4º ano do ensino fundamental. Como instrumentos, foram utilizados, além do Tafe, as tarefas de Bloco de Corsi, Teste de Trilhas A e B, Teste de Trilhas Pré-Escolares, Teste de Stroop Pré-Escolares e Subteste Dígitos da Escala Wechsler de Inteligência. Foram conduzidas análises estatísticas Kruskal Wallis para verificar a evidência de validade por relação com idade e análises de correlação não paramétrica de Spearman para avaliar as evidências de validade convergente. O instrumento investigado mostrou-se efetivo para discriminar entre as diferentes faixas etárias, assim, sensível ao desenvolvimento das FE. Os resultados obtidos no Tafe correlacionaram-se aos obtidos em outros testes que também avaliaram FE, mostrando claros padrões de convergência. Logo, as análises dos resultados fornecem evidências de validade ao Tafe, derivadas a partir de diferentes estratégias de investigação.(AU)

Executive functions (EF) refer to the set of skills that, in an integrated way, coordinate behavior and cognition. Thus, the commitment in developing EF is linked to several negative outcomes throughout life. Therefore, the assessment of these abilities in childhood is essential to identify and prevent possible harm in adult life. This study aimed to investigate evidences of validity of the Computerized Test for Evaluation of Executive Functions (TAFE) by the age criterion and by the pattern of correlation between TAFE measures and other EF measures. To this end, a sample of 51 children, between 4 and 10 years old, enrolled in a private school in the municipality of Goiânia (GO), from preschool to the 4th year of elementary school, was used. As instruments, in addition to TAFE, the tasks of Corsi Block, Trails Test A and B, Preschool Trails Test, Preschool Stroop Test and the Digits Subtest of the Wechsler Intelligence Scale were used. Kruskal Wallis statistical analyzes were conducted to verify the evidence of validity by age criteria and Spearman's nonparametric correlation analysis to assess evidence of convergent validation. The instrument investigated was effective to discriminate between different age groups, thus being sensitive to the development of EF. The results obtained in TAFE were correlated with those obtained in other tests that also assess FE, showing patterns of convergence. Therefore, the analysis of the results provides validation evidence to TAFE, derived from different investigation methods.(AU)

Las funciones ejecutivas (FE) se refieren al conjunto de habilidades que coordinan la conducta y la cognición. Así, el deterioro en el desarrollo de FE está relacionado con varios resultados negativos durante la vida. Por tanto, la valoración de estas habilidades en la infancia es fundamental para identificar y prevenir posibles daños en la vida adulta. Este estudio tuvo como objetivo investigar la evidencia de validez de la Prueba Computarizada para Evaluación de Funciones Ejecutivas (Tafe) por el criterio de edad y convergencia entre las medidas Tafe y otras medidas de FE. Para ello, se utilizó una muestra de 51 niños, de entre 4 y 10 años de edad, matriculados en un colegio privado de Goiânia (Goiás, Brasil) desde la preescolar hasta 4.º de primaria. Como instrumentos, además del Tafe, se utilizaron las tareas de Bloque de Corsi, el Test de los senderos A y B, el Test de los Senderos Preescolares, el Test de Stroop Preescolar y la Subprueba de Dígitos de la Escala Wechsler de Inteligencia. Se realizaron análisis estadísticos de Kruskal Wallis para verificar la validez por criterios de edad y análisis de correlación no-paramétrica de Spearman para evaluar la evidencia de validez convergente. Tafe demostró ser efectivo para discriminar entre diferentes grupos de edad, así es sensible al desarrollo de FE. Los resultados en Tafe se correlacionaron con los obtenidos en otras pruebas de FE, lo que muestra claros patrones de convergencia. Por tanto, el análisis aporta evidencias de validez para Tafe, derivadas de diferentes estrategias de investigación.(AU)

Sistema de avaliação de testes psicológicos: histórias contadas e não contadas / Psychological test evaluation system: stories told and untold / Sistema de evaluación de tests psicológicos: historias contadas y no contadas

O Sistema de Avaliação de Testes Psicológicos (SATEPSI) recebeu notoriedade entre brasileiros e estrangeiros por oferecer um complexo sistema de qualificação dos testes psicológicos, pouco visto em âmbito mundial. Sua elaboração dependeu de uma autarquia, que o financiou, normatizou e o mantém, mas também de pesquisadores docentes de avaliação psicológica, que trouxeram a expertise da área para que houvesse o pleno estabelecimento de seus parâmetros. Passadas duas décadas de seu lançamento, o SATEPSI foi tema de artigos, capítulos, lives e diálogos digitais, nos quais foram destaque, de modo geral, as Resoluções do Conselho Federal de Psicologia, que o normatiza, e seus impactos para a área de avaliação psicológica - como, por exemplo, o aumento do número de pesquisas e de testes brasileiros qualificados. O que se pretende neste artigo é mencionar sua construção, à luz dos autores que vivenciaram o SATEPSI em funções e tempos distintos. Atenção especial será dada aos Métodos Projetivos, cuja história ainda é pouco revelada.(AU)

The system to evaluate psychological tests (Satepsi) received notoriety among Brazilians and foreigners for offering a complex system of qualification of psychological tests, which is rarely seen worldwide. Its development depended on an autarchy (which financed, standardized, and maintains it) and on researchers teaching psychological assessment, who brought their expertise to the area so its parameters could be fully established. After two decades of its launch, Satepsi was the subject of articles, chapters, lives, and digital dialogues, which usually highlighted the Resolutions of the Federal Council of Psychology that normatize psychological evaluation and their impacts, such as the increase in the number of qualified Brazilian tests. This study aims to mention its construction in the light of the authors who experienced Satepsi in different functions and times, giving special attention to Projective Methods, whose history remains to be shown.(AU)

El Sistema de Evaluación de Tests Psicológicos (SATEPSI) ganó notoriedad entre los brasileños y los extranjeros por ofrecer un complejo sistema de calificación de los tests psicológicos, poco frecuente a nivel mundial. Su elaboración dependió de una autarquía, que lo financió, lo estandarizó y lo mantiene, pero también de investigadores docentes de evaluación psicológica, que trajeron la experiencia del área para que hubiera el pleno establecimiento de sus parámetros. Tras dos décadas de su lanzamiento, SATEPSI fue tema de artículos, capítulos, en directo y diálogos digitales, en los cuales destacaron, de modo general, las Resoluciones del Consejo Federal de Psicología que lo normatiza y sus impactos para el área de evaluación psicológica, como el aumento del número de investigaciones y de pruebas brasileñas calificadas. Lo que se pretende en este artículo es mencionar su construcción, a la luz de los autores que vivieron el SATEPSI en funciones y tiempos distintos. Se prestará especial atención a los métodos proyectivos cuya historia aún no se ha revelado.(AU)

20 anos da comissão consultiva em avaliação psicológica: desafios da ADI 3481 / 20 years of the advisory committee on psychological assessment: challenges of ADI 3481 / 20 años de la comisión consultiva en evaluación psicológica: desafíos de la ADI 3481

A Comissão Consultiva em Avaliação Psicológica (CCAP) do Conselho Federal de Psicologia (CFP), em seu 20º aniversário, vem discutir os possíveis efeitos, ainda efetivamente desconhecidos, da Ação Direta de Inconstitucionalidade (ADI) 3481, instruída no Supremo Tribunal Federal (STF), a qual desestruturou o modo como os testes psicológicos eram comercializados no Brasil. A livre comercialização de testes psicológicos coloca em risco a segurança de avaliações psicológicas e cabe à categoria profissional pensar estratégias de enfrentamento desses riscos. Neste artigo, são discutidos possíveis efeitos da ADI 3481 para a categoria profissional da psicologia, bem como para a sociedade em geral, e são também elencadas possíveis estratégias de enfrentamento desses riscos, sem desconsiderar aspectos éticos relacionados a eles. Dessa forma, busca-se neste manuscrito, além da problematização dos efeitos derivados da ADI 3481, pensar soluções ou alternativas que venham a redirecionar a trajetória da área da avaliação psicológica no Brasil. Com isso, abre-se um espaço de discussão e encaminhamentos que a categoria profissional precisará tomar nos próximos anos.(AU)

The Advisory Commission for Psychological Assessment of the Federal Council of Psychology discusses, on its 20th anniversary, the possible and still effectively unknown effects of the Direct Action of Unconstitutionality (DAU) 3481, following the Supreme Federal Court, which interrupted how psychological tests were marketed in Brazil. The free trade of psychological tests puts the safety of psychological assessments at risk, and this professional category must think of strategies to face these risks. This study discusses the possible effects of DAU 3481 for professional psychology and for society in general, listing possible strategies for coping with these risks without disregarding its ethical aspects. Thus, this study seeks to problematize the effects derived from DAU 3481 and think of solutions or alternatives that may redirect the trajectory of the field of psychological assessment in Brazil, thus opening a space for discussion and referrals professional psychology will require in the coming years.(AU)

La Comisión Consultiva en Evaluación Psicológica (CCEP) del Consejo Federal de Psicología (CFP), en su 20.º aniversario, propone discutir los posibles efectos aún efectivamente desconocidos de la Acción Directa de Inconstitucionalidad (ADI) 3481, determinada por el Supremo Tribunal Federal (STF), por la cual trastornó la forma de comercializar las pruebas psicológicas en Brasil. La comercialización sin restricciones de las pruebas psicológicas pone en riesgo la seguridad de las evaluaciones psicológicas, y le corresponde a la categoría profesional pensar estrategias para enfrentar estos riesgos. En este artículo se discuten los posibles efectos de la ADI 3481 para la categoría profesional de la Psicología, así como para la sociedad en general, pero también se enumeran posibles estrategias para el enfrentamiento de estos riesgos, sin descuidar los aspectos éticos relacionados con ellos. Así, este manuscrito busca, además de problematizar los efectos derivados de la ADI 3481, pensar en soluciones o alternativas que puedan reconducir la trayectoria del campo de la evaluación psicológica en Brasil. Esto abre un espacio de discusión y derivaciones que la categoría profesional deberá tomar en los próximos años.(AU)

Reflexões sobre o trabalho on-line da neuropsicologia infantil numa visão histórico-cultural / Reflections on the online work of child neuropsychology from a cultural-historical perspective / Reflexiones sobre el trabajo online de la neuropsicología infantil desde una perspectiva histórico-cultural

O isolamento social provocado pela pandemia de covid-19 exigiu uma intensificação no estabelecimento de parâmetros para o trabalho do neuropsicólogo com suas especificidades. Assim, o objetivo deste artigo é levantar reflexões sobre a prática neuropsicológica infantil via internet baseada na nossa experiência prática vivenciada no período, relacionando com as referências teóricas da abordagem histórico-cutural e outros trabalhos que discorrem sobre as particularidades dessa modalidade de atendimento. Para isso, foi realizada uma pesquisa bibliográfica em fontes reconhecidas, como SciELO e Google Acadêmico, utilizando-se marcadores como "neuropsicologia on-line" e "teleneuropsicologia", entre outros, em português, inglês e espanhol. Os trabalhos encontrados foram utilizados como fundamento para discutir as ideias aqui apresentadas, a partir da experiência dos autores durante esse período. Considera-se que os neuropsicólogos precisam enfrentar a possível necessidade do trabalho on-line, considerando que as situações vividas durante a pandemia tendem a aumentar a necessidade de avaliações e intervenções neuropsicológicas. Ao mesmo tempo, se prevê que essa via de trabalho deve se intensificar após o isolamento, dentro do que se tem denominado como "o novo normal". Para isso, é essencial que haja esforço por parte da comunidade neuropsicológica para mostrar como essa área de trabalho pode ser útil para crianças, seus pais e professores. Conclui-se que é preciso ter a noção de que nenhuma via ou metodologia de trabalho é absolutamente completa, sendo essencial a instrumentalização das vantagens e desvantagens de todas elas.(AU)

The social isolation caused by the COVID-19 pandemic demanded intensification in the establishment of parameters for the neuropsychologist's work with its specificities. Thus, the aim of this article is to raise reflections on child neuropsychological practice via the internet, based on our practical experience in this period, relating to the theoretical references of the cultural-historical approach and other works that discuss the particularities of this type of care. For this, a bibliographic research was carried out in recognized sources such as SciELO and Google Scholar, using markers such as "online neuropsychology" and teleneuropsychology, among others, in Portuguese, English and Spanish. The works found were used as a basis for discussing the ideas presented here which are based on the authors' experience during this period. It is considered that neuropsychologists need to face the possible need for online work, considering that situations experienced during the pandemic tend to increase the need for neuropsychological assessments and interventions. At the same time, it is expected that this work path should intensify after isolation, within of what has been called "the new normal". For this, an effort by the neuropsychological community is essential to show how this area of work can be useful for children, their parents and teachers. It is concluded that it is necessary to have the notion that no way or work methodology is absolutely complete, being essential the instrumentalization of the advantages and disadvantages of all of them.(AU)

El aislamiento social provocado por la pandemia de Covid-19 exigió una intensificación en el establecimiento de parámetros para el trabajo del neuropsicólogo con sus especificidades. Así, el objetivo de este artículo es plantear reflexiones sobre la práctica neuropsicológica infantil a través de internet, a partir de nuestra experiencia práctica en este período, en relación con los referentes teóricos del enfoque histórico-cultural y otros trabajos que discuten las particularidades de este tipo de cuidado. Para esto, se realizó una investigación bibliográfica en fuentes reconocidas como SciELO y Google Académico, utilizando marcadores como "neuropsicología en línea" y teleneuropsicología, entre otros, en portugués, inglés y español. Los trabajos encontrados sirvieron de base para discutir las ideas aquí presentadas a partir de la experiencia de los autores durante este período. Se considera que los neuropsicólogos necesitan enfrentar la posible necesidad de trabajo en línea, considerando que las situaciones vividas durante la pandemia tienden a incrementar la necesidad de evaluaciones e intervenciones neuropsicológicas. Al mismo tiempo, se espera que esta vía de trabajo se intensifique después del aislamiento, dentro de lo que se ha denominado "la nueva normalidad". Para esto, un esfuerzo de la comunidad neuropsicológica es fundamental para mostrar cómo esta área de trabajo puede ser útil para los niños, sus padres y profesores. Se concluye que es necesario tener la noción de que ninguna vía o metodología de trabajo es absolutamente completa, siendo imprescindible la instrumentalización de las ventajas y desventajas de todas ellas.(AU)

An assessment of the impact of formal preparation activities on performance in the University Clinical Aptitude Test (UCAT): a national study.

BACKGROUND: Previous studies have shown performance in the University Clinical Aptitude Test (UCAT) to be associated with measures of candidate socio-economic advantage such as parental occupation and type of school attended. It is possible that access to preparation support and materials may in part explain these associations. In this paper we determine whether use of formal preparation resources is associated with higher UCAT scores and whether differences in use of preparation resources exist between socio-demographic groups. METHODS: After completing the 2017 UCAT UK school-leaver candidates (n = 14,332) were asked to answer a questionnaire regarding their use of official UCAT and commercial resources, school-based support, and time spent preparing. Multiple linear and logistic regression models were used to evaluate the associations between preparedness, demographic characteristics and UCAT performance. RESULTS: Five thousand, four hundred thirty-nine (38%) candidates responded to the questionnaire. Use of freely available UCAT official practice tests, paid commercial materials, attendance at school-based preparation courses and spending more time preparing were significantly associated with higher UCAT scores. Candidates who were from less deprived backgrounds and attending independent or grammar schools were significantly more likely to use paid commercial materials and spend longer preparing. CONCLUSIONS: Reported use of preparation resources varies between candidates from different socio-demographic backgrounds and is associated independently with performance in the UCAT. Increasing the availability of freely available resources may mitigate some of these differences.

BMAT's predictive validity for medical school performance: A retrospective cohort study.

BACKGROUND: Although used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools. METHODS: This was a retrospective cohort study of four ICSM (1188 students, entering 2010-2013) and three LKCMedicine cohorts (222 students, 2013-2015). We investigated associations between BMAT Section 1 ('Thinking Skills'), Section 2 ('Scientific Knowledge and Applications') and Section 3a ('Writing Task') scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A-levels) in a subset of ICSM students. RESULTS: When BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08-0.19; LKCMedicine: 0.22-0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09-0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A-level grades. DISCUSSION: Schools who wish to assess scientific knowledge independently of A-levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration.

Does Inclusion of Mental Agility Test in Admission Exam Help Better Student Selection in the Health Professions Education Programs?

Academic ability test has been used predominantly in student selection of medical and allied health profession education programs in Nepal. But the use of academic performance as the single selection criterion puts the students from low socioeconomic background at greater disadvantage despite equal suitability due to the lack of adequate guidance and support during their schooling. To address this limitation, use of aptitude test i.e. both the general cognitive and non-cognitive ability tests that measures fluid intelligence and personality traits respectively has been practiced. The measurement of non-cognitive traits has been found to predict the clinical examination score. In Nepal, for the first time, Patan Academy of Health Sciences implemented the assessment of aptitude test (both cognitive and non-cognitive ability test) for the student selection in the undergraduate medical program. Since the inception, Medical Education Commission in Nepal embraced Mental Agility Test, a component of an aptitude test, along with the academic ability test for the nationwide common entrance test in all the undergraduate Health Professions Education Programs. This indeed is an innovative approach in student selection, but in the context of Nepal whether the use of these tools is appropriate in the entrance exam requires psychometric evaluation and further validation through graduates' performance after their enrolment. Keywords: Health professions education; mental agility test; student selection.

Association of admissions factors with student performance in preclinical operative dentistry.

PURPOSE/OBJECTIVE: Admission to dental school is based on various factors including academic achievements in undergraduate coursework and the Dental Admission Test (DAT) scores. Students' success in an operative course requires fundamental knowledge, hand skills, spatial awareness, and self-assessment ability. The goal of this study is to evaluate how admissions factors, such as Grade Point Average (GPA) and DAT, including the Perceptual Ability Test (PAT), relate to students' academic and preclinical performance and self-assessment skills in preclinical operative dentistry. METHODS: A total of 239 students were included from seven class years (2016-2022). Third-year dental students participated in a preclinical operative dentistry course. At the end of the course, they took the final multiple-choice exam and performed four competency examination procedures: Class II amalgam preparation and restoration and Class III resin-composite preparation and restoration. Calibrated faculty graded students' work independently and students also self-assessed their performance using the same rubrics as faculty. Linear regressions were performed to estimate the association between the admission factors with the mean faculty scores (measuring preclinical performance), student-faculty (S-F) gap scores (evaluating self-assessment skills), and their final didactic exam scores. RESULTS: Overall, students' self-assessment was higher compared to faculty score. Linear regression analysis demonstrated positive correlations between the PAT and students' preclinical performance as well as between the DAT and their didactic exam scores. In general, S-F gap score decreased as PAT score increased, and it was statistically significant lower for the Class III preparation, indicating a better self-assessment skill. No correlations were observed between student performance and GPA scores. CONCLUSION: The findings from the association between student performance and admission factors may play an important role in the dental school admissions process and assist students who may benefit from early faculty intervention and support.

Factors associated with declaration of disability in medical students and junior doctors, and the association of declared disability with academic performance: observational study using data from the UK Medical Education Database, 2002-2018 (UKMED54).

OBJECTIVES: To examine factors associated with declaration of disability by medical students and doctors, and the association of declared disability with academic performance. DESIGN: Observational study using record-linked data collected between 2002 and 2018. SETTING: UK Medical Education Database is a repository of data relating to training of medical students and doctors. Disability and other data are record-linked. PARTICIPANTS: All students starting at a UK medical school between 2002 and 2018 (n=135 930). MAIN OUTCOME MEASURES: Declared disability was categorised by the Higher Education Statistics Authority. Outcomes related to undergraduate academic performance included scores in the educational performance measure (EPM), prescribing safety assessment and situational judgement test. Performance in postgraduate examinations was studied, as well as prior attainment in school examinations and aptitude tests. RESULTS: Specific learning disability (SLD) was the most commonly declared disability (3.5% compared with the next most commonly declared disability at 1.0% of n=129 345 all cases in the study), and during the period covered by the data, SLD declarations increased from 1.4% (n=6440 for students starting in 2002) to 4.6% (n=8625 for students starting in 2018). In a logistic regression, the following factors predicted recording of SLD on entry to medical school ((exp(B)±95% CI), p<0.0001 unless otherwise stated): attendance at a fee-paying school (2.306±0.178), graduate status (1.806±0.205), participation of local areas quintile (1.089±0.030), age (1.034±0.012). First year medical students were less likely to declare SLD if they were from a non-white ethnic background (Asian/Asian British 0.324±0.034, black/black British 0.571±0.102, mixed 0.731±0.108, other ethnic groups 0.566±0.120), female (0.913±0.059; p=0.007) or from a low index of multiple deprivation quintile (0.963±0.029); p=0.017. In univariate analysis with Bonferroni corrections applied for multiple tests, no significant difference was observed in the recording of SLD according to socioeconomic class (χ2=5.637, p=1), whether or not a student's parents had a higher education (χ2=0.140, p=1), or whether or not a student had received a United Kingdom Clinical Aptitude Test (UKCAT) bursary (χ2=7.661, p=0.068). Students who declared SLD at some point in medical school (n=4830) had lower EPM normalised deviate values (-0.390) than those who did not (-0.119) (F=189.872, p<0.001). Those for whom SLD was recorded were as likely to complete the course successfully as those who did not declare disability (93.0% successful completion by those for whom SLD declared from year 1 (n=2480), 92.2% by those for whom SLD declared after year 1 (n=2350), 91.6% by those for whom SD not declared at any point (n=85 180)) (χ2=6.905, p=0.032). Of 3580 first year students who declared SLD, 43.1% had not sat the UKCAT Special Educational Needs aptitude test (which gives extra time for those with special educational needs), while 28% of 2400 registrants for whom SLD was recorded as medical students did not declare it at General Medical Council registration. CONCLUSIONS: Substantial increases in declaration of SLD may reflect changes in the social and legal environment during the period of the study. Those who declare SLD are just as likely to gain a primary medical qualification as those who do not. For some individuals, disability declaration appears to depend on context, based on differences in numbers declaring SLD before, during and after medical school.

Situational judgement test performance and subsequent misconduct in medical students.

INTRODUCTION: Situational judgement tests (SJTs) have been widely adopted, internationally, into medical selection. It was hoped that such assessments could identify candidates likely to exhibit future professional behaviours. Understanding how performance on such tests may predict the risk of disciplinary action during medical school would provide evidence for the validity of such SJTs within student selection. It would also inform the implementation of such tests within student recruitment. METHODS: This cohort study used data for 6910 medical students from 36 UK medical schools who sat the University Clinical Aptitude Test (UCAT) SJT in 2013. The relationship between SJT scores at application and the risk of subsequent disciplinary action during their studies was modelled. The incremental ability of the SJT scores to predict the risk of disciplinary action, above that already provided by UCAT cognitive test scores and secondary (high) school achievement, was also evaluated in 5535 of the students with information available on this latter metric. RESULTS: Two hundred and ten (3.05%) of the students in the cohort experienced disciplinary action. The risk of disciplinary action reduced with increasing performance on the admissions SJT (odds ratio (OR) 0.80, 95% confidence interval (CI) 0.69 to 0.92, p = 0.002). This effect remained similar after adjusting for cognitive performance and prior academic attainment (OR 0.77, 95% CI 0.65 to 0.92, p = 0.004). The overall estimated effect-size was small (Cohen's d = 0.08) and no evidence of 'threshold' effects were observed for the SJT scores and risk of disciplinary action. CONCLUSIONS: Performance on admissions SJTs can, at least modestly, incrementally predict the risk of subsequent disciplinary action, supporting their use in this context. However, for this SJT and outcome, there did not seem a distinct threshold score above which the risk of disciplinary action disproportionately increased. This should be considered when using the scores within medical selection.

UCAT and dental student selection in the UK - what has changed?

Introduction The University Clinical Aptitude Test (UCAT) has been used since 2006 by a consortium of UK medical and dental schools to assist in undergraduate selection. In 2019, UCAT was used by 30 universities (14 dental schools).Aim To report how UCAT use has changed in undergraduate student selection in the UK.Methods UCAT use was categorised and trends identified from annual telephone interviews with dental school admission tutors; this process started in 2011.Results Dental schools using UCAT rose from 8 (2006) to 14 (2020). The most significant use of the test to select applicants for interview was as a weighted factor; at offer stage, UCAT was most used to discriminate between applicants at borderlines. A growing number of dental schools are using the Situational Judgement Test (SJT) in selection (2019, n = 6). In 2019, eight schools adjusted selection processes for widening access applicants. Multiple mini interviews are now used by the majority (n = 10) of dental schools.Conclusions UCAT represents a significant factor in selection to UK undergraduate dental programmes and is used by all but two dental schools. In most schools, UCAT contributes in a substantial way to selection outcomes and strength in test use has grown over time.

Can achievement at medical admission tests predict future performance in postgraduate clinical assessments? A UK-based national cohort study.

OBJECTIVE: To determine whether scores on two undergraduate admissions tests (BioMedical Admissions Test (BMAT) and University Clinical Aptitude Test (UCAT)) predict performance on the postgraduate Membership of the Royal Colleges of Physicians (MRCP) examination, including the clinical examination Practical Assessment of Clinical Examination Skills (PACES). DESIGN: National cohort study. SETTING: Doctors who graduated medical school between 2006 and 2018. PARTICIPANTS: 3045 doctors who had sat BMAT, UCAT and the MRCP. PRIMARY OUTCOME MEASURES: Passing each section of the MRCP at the first attempt, including the clinical assessment PACES. RESULTS: Several BMAT and UCAT subtest scores displayed incremental predictive validity for performance on the first two (written) parts of the MRCP. Only aptitude and skills on BMAT (OR 1.34, 1.08 to 1.67, p=0.01) and verbal reasoning on UCAT (OR 1.34, 1.04 to 1.71, p=0.02) incrementally predicted passing PACES at the first attempt. CONCLUSIONS: Our results imply that the abilities assessed by aptitude and skills and verbal reasoning may be the most important cognitive attributes, of those routinely assessed at selection, for predicting future clinical performance. Selectors may wish to consider placing particular weight on scales assessing these attributes if they wish to select applicants likely to become more competent clinicians. These results are potentially relevant in an international context too, since many admission tests used globally, such as the Medical College Admission Test, assess similar abilities.

An investigation of the predictive validity of selection tools on performance in physiotherapy training in Australia.

OBJECTIVES: Despite a large body of research on selection in medical education, very little is conducted in other health professions. This study investigated the predictive validity of multiple selection tools on academic and clinical performance outcomes of undergraduate physiotherapy students. DESIGN: A retrospective observational study. SETTING: Undergraduate physiotherapy program in Australia. PARTICIPANTS: 497 undergraduate physiotherapy students across seven entry cohorts. Including students directly from secondary school (n=381) and with prior tertiary study (n=116). MAIN OUTCOME MEASURES: Academic performance as measured by written examinations. Clinical performance, measured by Objective Structured Clinical Examinations (OSCEs) during on-campus units and the Assessment of Physiotherapy Practice (APP) for off-campus clinical placements. Predictor variables included selection tools (academic achievement, interview, aptitude test) and demographic variables (age, gender). RESULTS: Selection interview was a positive predictor of OSCEs and final year clinical performance in direct school leaver participants. Academic achievement scores from selection positively predicted written examinations scores. CONCLUSION: Clinical and academic performance were predicted by tools measuring different domains at selection. Assessing broadly across academic and non-academic domains at selection can be valuable in identifying applicants who will be able to meet the range of outcomes for course completion and subsequent registration in the physiotherapy profession.

Assessing the predictive validity of the UCAT-A systematic review and narrative synthesis.

BACKGROUND: The University Clinical Aptitude Test (UCAT) is an admissions assessment used by a consortium of universities across the UK, Australia, and New Zealand, to aid the selection of applicants to medical and dental degree programmes. The UCAT aims to measure the mental aptitude and professional behaviours required to become successful doctors and dentists. We conducted a systematic review to establish the predictive value of the UCAT across measures of performance at undergraduate and post-graduate levels. METHODS: A literature search was conducted in April 2020 using eight electronic databases: MEDLINE, APA PsycInfo, SCOPUS, Web of Science, EThOS, OpenGrey, PROSPERO, and the UCAT website. Data were extracted from selected studies and tabulated as results matrices. A narrative synthesis was performed. RESULTS: Twenty-four studies satisfied our inclusion criteria, 23 of which were deemed to be of good quality (using the Newcastle-Ottawa Scale). For over 70% of univariate data points, the UCAT exerted no statistically significant predictive validity; for the remainder, predictive power was weak. The cognitive total and verbal reasoning subtests had the largest evidence base as weakly positive predictors of academic performance. The SJT subtest was a weak predictor of professional behaviour during medical school. Studies specific to dental schools demonstrated variable findings across the five studies. Only 1 study looked at post-graduate outcome measures and demonstrated that the UCAT was not a predictor of health- or conduct-related fitness to practice declarations at GMC registration. CONCLUSIONS: These data provide some support for the use of cognitive total and verbal reasoning subtests as part of medical school selection. Further research is needed to investigate outcomes beyond professional registration and for dental students.